Tuning the configuration of a service on a computer system to handle a desired load is typically a manual process. In other words, a tuning process typically involves a user manually tweaking different attributes of the service or the underlying system in the hopes of improving the performance of the system running the service. The attributes to be tweaked may be as specific as the numbers of threads in different thread pools but may include any aspect of the service that might affect performance or throughput. Multiple attributes may need to be manually modified many times in an effort to improve the performance significantly, especially if performance is dependent on multiple interrelated attributes. For heterogeneous multi-host web services that have specific targets in terms of throughput, latency, or stability, the tuning process may be especially complex and time-consuming.
A typical approach to this manual tuning process involves trial and error. A user may making some initial guesses on optimal values, put the service into production based on the guesses, and manually analyze the load it can handle. The user may then tweak the values even further, again based on guesswork. In some circumstances, parts of the system will change dramatically over time, thus making the original estimates outdated. However, because this approach to tuning is manual and time-consuming, the tuning may not be performed on a regular basis. As a result, outdated and inefficient settings may remain in place until they have significantly adverse effects on performance. When performance estimates are outdated or entirely absent, hardware resources may be wasted on systems that are not operating optimally. For example, in a fleet of 10,000 hosts, a 10% improvement in throughput can mean a savings of 1000 hosts as well as savings in resources such as network bandwidth and power consumption.
Accordingly, it is desirable to have efficient techniques for tuning services.
While embodiments are described herein by way of example for several embodiments and illustrative drawings, those skilled in the art will recognize that embodiments are not limited to the embodiments or drawings described. It should be understood, that the drawings and detailed description thereto are not intended to limit embodiments to the particular form disclosed, but on the contrary, the intention is to cover all modifications, equivalents and alternatives falling within the spirit and scope as defined by the appended claims. The headings used herein are for organizational purposes only and are not meant to be used to limit the scope of the description or the claims. As used throughout this application, the word “may” is used in a permissive sense (i.e., meaning “having the potential to”), rather than the mandatory sense (i.e., meaning “must”). Similarly, the words “include,” “including,” and “includes” mean “including, but not limited to.”